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(54) Method for correcting luminance gradation in an image pickup apparatus 



(57) A synthesized picture signal of a dynamic im- 
age is produced by combining a long-time picture signal 
representing a low luminance and a short-time picture 
signal representing a high luminance, pieces of histo- 
gram data corresponding to a plurality of luminance lev- 
els are detected from the synthesized picture signal, a 
sum of values of pieces of histogram data is repeatedly 
calculated as a summed value of a piece of summed 
histogram data while the number of pieces of histogram 
data is increased, and an inclination of a line indicated 
by a series of summed values is detected as a lumi- 
nance gradation characteristic. Thereafter, the lumi- 
nance gradation characteristic is corrected to a correct- 
ed luminance gradation characteristic according to a re- 
lationship between a maximum luminance level and an 
upper limit luminance level of a dynamic range, and a 
luminance gradation corrected signal composed of the 
pieces of histogram data corresponding to gradation 
corrected luminance levels is produced according to the 
corrected luminance gradation characteristic to make a 
maximum gradation corrected luminance level equal to 
or lower than the upper limit luminance level. Therefore, 
a low luminance object and/or a high luminance object 
can be clearly reproduced within the dynamic range. 
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Description 

The present Invention relates generally to a luminance gradation correcting method in an image pickup apparatus 
used for a video camera or the like, and more particularly to a luminance gradation correcting method tor reproducing 
5 a dynamic image of an object composed of a light object portion and/or a dark object portion at a natural feeling even 
though a dynamic range of a displaying unit of an image pickup apparatus for luminance is narrow 

In a single-plate type color camera, a ratio of the highest reproducible luminance and the lowest reproducible 
luminance is two or three, so lhal a dynamic range of the single-plale type color camera is about two or three limes. 
However, because the luminance in an outdoor place is about 50000 lux on a sunny day and the luminance in a room 
io is about 500 lux, so that a dynamic range of about 100 times is required for the single-plate type color camera in cases 
where a light object portion placed in the sunny outdoor place and a dark object portion placed in a room are simulta- 
neously photographed. To photograph a particular object composed of a light object portion and a dark object portion 
as a dynamic image by using an image pickup apparatus of a narrow dynamic range, a new charge coupled device 
(CCD) operated at a transmission speed two times as fast as that in a conventional CCD is provided for the image 
15 pickup apparatus, a first picture signal (called a long-time picture signal) is obtained by photographing the particular 
object at a long exposure time (for example, about 1/60 second which is the same as that in an image pickup apparatus 
with the conventional CCD), a second picture signal (called a short-time picture signal) is obtained by photographing 
the particular object at a short exposure time (for example, about 1/1000 second), the long-time picture signal and the 
short-time picture signal are combined to produce a synthesized picture signal for each field period, and a luminance 
20 gradation in the synthesized picture signal is corrected to make a corrected luminance of an image reproduced by the 
synthesized picture signal correspond to an actual luminance of the particular object and to set the corrected luminance 
of the reproduced image within the narrow dynamic range of the image pickup apparatus. In this case, an image of 
the dark object portion is clearly indicated by the long-time picture signal though an image of the light object portion is 
not clearly indicated by the long-time picture signal, and an image of the light object portion is clearly indicated by the 
25 short-time picture signal though an image of the dark object portion is not clearly indicated by the short-time picture 
signal. Therefore, even though the dynamic range of the image pickup apparatus with the new CCD is narrow, the dark 
object portion and the light object portion in the particular object can be clearly indicated by the synthesized picture 
signal in cases where the luminance gradation in the synthesized picture signal is properly corrected. 

An image pickup apparatus operated according to a conventional luminance gradation correcting method is de- 
30 scribed with reference to Fig. 1 

Fig. 1 is a block diagram of an image pickup apparatus operated according to a conventional luminance gradation 
ccrrecting method. 

As shown in Fig. 1 , an image pickup apparatus 11 is composed of 

an image pickup device 12 having IvTN pixels (M pixels in the lateral direction and N pixels in the longitudinal 
direction) for producing a long-time analog picture signal corresponding to a long exposure time and a short-time 
analog picture signal corresponding to a short exposure time by alternately performing the photographing of a 
particular object composed of a light object portion and a dark object portion at a long exposure time and the 
photographing of the particular object at a short exposure time and transmitting the long-time analog picture signal 
and the short -time analog picture signal in a pair for each field period; 

a pre-processing unit 13 for performing a correlated double sampling and a automatic gain control for the long- 
time analog picture signal and the short-time analog picture signal and amplifying the long-time analog picture 
signal and the short-time analog picture signal for each field period; 

an ana log -digital (AID) converter 14 for converting the long-time analog picture signal and the short-time analog 
picture signal into a long-time digital picture signal and a short-time digital picture signal for each field period; 
a time-axis converter 1 5 for producing the long-time digital picture signal and the short-time digital picture signal 
of which transmission time periods (or time axes) agree with each other by alternately storing the long-time digital 
picture signal and the short-time digital picture signal in a long-time signal memory and a short-time signal memory 
for each field period and simultaneously reading out the long-time digital picture signal and the short-time digital 
picture signal from the signal memories; 

a luminance level synthesizing unit 16 lor combining the long-time digital picture signal and the short-time digital 
picture signal to produce a synthesized picture signal having a luminance range, luminance values of the pixels 
in the luminance range being classified into 16 luminance levels L(i) (i = 0 to 15), the 16 luminance levels being 
determined by equally dividing the luminance range into 16 divided luminance ranges, each luminance level rep- 
resenting luminance values of one divided luminance range having the same divided luminance width Lw, and the 
synthesized picture signal being classified into 16 pieces of histogram data of values H(i) corresponding to the 1 6 
luminance levels; 

a histogram data detecting unit 17 for detecting the values H(i) of the pieces of histogram data, the value H(i) of 
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each piece of histogram data indicating the number of pixels having one luminance level, and the number of pixels 
having one luminance level being called a luminance frequency; 

a luminance gradation characteristic producing unit 18 S realized by a microcomputer lor normalizing the values H 
(i) of the pieces of histogram data to produce values Hnl(i) of pieces of normalized histogram data, replacing the 

5 values Hnl (i) with values Hn2(i) of pieces of limited histogram data to limit an inclination of a curved line indicated 

by a series of summed values Hns1 (j) = £Hn1 (i) (i = 0 to j, j = 0 to 1 5), summing up the values Hn2(0), Hn2(1 ) : ~ 
and Hn2(j) to produce a plurality of summed values Bns2(j) = IHn2(i) of pieces of summed histogram data and 
performing a normalization processing for the pieces of summed histogram data to produce pieces of summed 
normalized histogram data having values Hnsn(j), an inclination of a curved line indicated by a series of summed 

10 values Hnsn(j) of the pieces of summed normalized histogram data denoting a luminance gradation characteristic 

of a synthesized image indicated by the synthesized picture signal; 

a luminance gradation correcting unit 19 for correcting a luminance gradation of the synthesized image indicated 
by the synthesized picture signal according to the luminance gradation characteristic to emphasize a luminance 
contrast between a particular luminance level of a high luminance frequency and a group of neighboring luminance 
levels adjacent to the particular luminance level and producing a luminance gradation corrected picture signal, the 
luminance gradation corrected picture signal being classified into 1 6 pieces of gradation corrected histogram data 
of values H(i) corresponding to 16 gradation corrected luminance levels; 

a main picture signal processing unit 20 for processing the luminance gradation corrected picture signal to produce 
a luminance gradation corrected video signal: 

a displaying unit 21 for reproducing a gradation corrected image of the particular object according to the luminance 
gradation corrected video signal, a ratio of the highest corrected luminance level to the lowest corrected luminance 
level being within a narrow dynamic range of the displaying unit 21. 
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In the above configuration, the image pickup device 12 is made of a CCD operated at a transmission speed two 
times as fast as that in a normal CCD, a particular object portion composed of a light object portion and a dark object 
portion is photographed by the image pickup device 12 at a long exposure time (1/64 second) to produce a long-time 
analog picture signal, and the particular object is photographed at a short exposure time (1/1000 second) to produce 
a short-time analog picture signal. The long-time analog picture signal and the short-time analog picture signal are 
alternately produced in the image pickup device 12, and the long-time analog picture signal and the short-time analog 
picture signal are transmitted in a pair for each field period. Thereafter, the long-time analog picture signal and the 
short-time analog picture signal are processed in the pre-processing unit 13 and are converted into a long-time digital 
picture signal and a short -time digital picture signal in the analog-digital converter 14. Thereafter, the transmission 
timing of the long-time digital picture signal is made agree with that of the short-time digital picture signal in the time- 
axis converter 15, and the long-time digital picture signal and the short-time digital picture signal transmitted at the 
same time are combined to produce a synthesized picture signal in the luminance level synthesizing unit 16. 

Fig. 2 shows a combining method for producing one synthesized picture signal from one long-lime digital picture 
signal and one short-time digital picture signal. 

In Fig. 2, an X-axis indicates a luminance value of the long-time digital picture signal for each low luminance pixel 
and a luminance value of the short-time digital picture signal for each high luminance pixel, and a Y-axis indicates a 
40 luminance value of a synthesized picture signal at the corresponding pixel. A low luminance combination level L1 
corresponding to the long-time digital picture signal and an upper luminance combination level L2 corresponding to 
the short-time digital picture signal are set, the long-time digital picture signal is adopted as the synthesized picture 
signal for pixels having luminance values equal to or lower than the low luminance combination level L1 because an 
image of a dark object portion can be reproduced well by the long-time digital picture signal even though the dark 
object portion is photographed at a long exposure time, and the short -time digital picture signal is used for pixels having 
luminance values equal to or higher than the upper luminance combination level L2 because an image of a light object 
portion can be reproduced well by the short-time digital picture signal even though the light object portion is photo- 
graphed at a short exposure time. 

Because the short-time digital picture signal is produced at an exposure time shorter than that in the long-time 
50 digital picture signal, a luminance value Yshort of the short-time digital picture signal at one pixel is lower than a 
luminance value Ylong of the long-time digital picture signal at the same pixel, so that an offset value Yoffset is added 
to a luminance value Yshort of the short-time digital picture signal for each pixel. Therefore, luminance values of the 
synthesized picture signal for pixels having luminance values equal to or lower than the low combination level L1 are 
expressed by Ylong, and luminance values of the synthesized picture signal for pixels having luminance values equal 
to or higher than the upper combination level L2 are expressed by Yshort + Yoffset. A low synthesized luminance level 
L3 of the synthesized picture signal corresponds to the low luminance combination level L1 of the long-time digital 
picture signal, and an upper synthesized luminance level L4 of the synthesized picture signal corresponds to the upper 
luminance combination level L2 of the shorMime digital picture signal. The low synthesized luminance level L3 is 
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usually set to half an upper limit luminance value Lmax of the dynamic range of the displaying unit 21 . 

Also, a gain K linearly changing in a transition luminance range between the levels L1 and L2 is used on condition 
that the K=1 is satisfied at the low combination level L1 and K=0 is satisfied at the upper combination level L2, and 
luminance values of the synthesized picture signal tor pixels in the transition luminance range are expressed by K'YIong 

5 + (1 -K)*(Yshort + Yoffset). Therefore, the influence of the short-time digital picture signal is increased in the synthesized 
picture signal as the luminance value is increased in the transition luminance range. 

Therefore, a luminance value Ymix of the synthesized picture signal is determined for each pixel. 
Because the synthesized picture signal has a plurality of luminance values corresponding to the M*N pixels, in 
cases where the luminance values are equally classified into i 6 luminance levels, 1 6 pieces of histogram data of values 

10 H(i) (i = 0 to 15) corresponding to the 16 luminance levels can be obtained. In this case, the value H(i) of each piece 
of histogram data indicates the number of pixels having one luminance level and is called luminance frequency. Also, 
because the synthesized picture signal for pixels having luminance values equal to orhigher than the uppercombination 
level L2 has the luminance values Yshort + Yoffset higher than the luminance values Yshort, the highest luminance 
level L(15) exceeds an upper limit luminance value of the dynamic range of the displaying unit 21, so that a ratio of 

15 the highest luminance level to the lowest luminance level in the synthesized picture signal exceeds a dynamic range 
of the displaying unit 21 . 

Thereafter, the values H(i) of the pieces of histogram data are detected by the histogram data detecting unit 17, 
and a luminance gradation characteristic of a synthesized image indicated by the synthesized picture signal is produced 
by the luminance gradation characteristic producing unit 18. 
20 Fig. 3 shows a flow chart of a method for producing a luminance gradation characteristic of a synthesized image 

indicated by the synthesized picture signal. 

As shown in Fig. 3, a sum Hall of the values H(i) of the pieces of histogram data is calculated in a step S1 01 . 

25 15 

Ball = £H(i) 
i-o 

Thereafter, the pieces of histogram data are normalized to produce values Hn1(i) of pieces of normalized histogram 
30 data in a step S 102. 

Hn1(i)= H(n)/Hall 

35 Thereafter, assuming that a plurality of summed values Hns1 (j) = ZHn1 (i) (j = 0 to 1 5) are produced from the normalized 
histogram data as a luminance gradation characteristic and a luminance contrast of the synthesized image indicated 
by the synthesized picture signal is emphasized according to the luminance gradation characteristic, a slight difference 
between luminance levels of high luminance frequencies is excessively emphasized, so that a signal to noise (S/N) 
ratio deteriorates or a luminance contrast between luminance levels of low luminance frequencies is lost. To prevent 

40 this drawback, an inclination of a curved line indicated by a series of summed values Hns1(j) is limited. That is, in a 
step S103, in cases where one value Hn1(i) of the normalized histogram data is lower than a low limitation value Cpmin 
(i), a value Hn2(i) of a piece of limited histogram data corresponding to the value Hn1 (i) is set to the low limitation value 
Cpmin(i). Also, in cases where one value Hn1(i) of the normalized histogram data is higher than an upper limitation 
value Cpmax(i), a value Hn2(i) of a piece of limited histogram data corresponding to the value Hn1 (i) is set to the upper 

45 limitation value Cpmax(i). Also, in cases where one value Hn1(i) of the normalized histogram data is ranged between 
the limit values Cpmin(i) and cpmax(i), the value Hn1(i) is used as a value Hn2(i) of a piece of limited histogram data. 

Thereafter, in a step SI 04, a sum of the values Hn2(0), Hn1 (I),— and Hn1 0) of the pieces of limited histogram data 
is calculated for each variable value j (j = 0 to 1 5). 

so 

}. 

Hns2 ( j ) = lHn2(i) 

55 

Therefore, a plurality of summed values Hns2(j) of pieces of summed histogram data are obtained. Thereafter, 
because the highest luminance level L(15) of the synthesized picture signal exceeds. an upper limit luminance value 
of the dynamic range of the displaying unit 21, a normalization processing is performed for the pieces of summed 
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histogram data in a step S105 by multiplying the value of each pieces of summed histogram data by a constant value 
K1 to make the highest luminance level L(15) of the synthesized picture signal agree with an upper limit luminance 
value Lrriax of the dynamic range of the displaying unit 21. 

5 

Hnsn(j) = Hns2(j) * K1 
K1 = Lmax/Hns2(15) 

10 

Therefore, a plurality of summed values Hnsn(j) of pieces of summed normalized histogram data are obtained. In 
this case, an inclination of a curved line indicated by the series of summed values Hnsn(j) of the pieces of summed 
normalized histogram data denotes a luminance gradation characteristic of a synthesized image indicated by the syn- 
thesized picture signal, so that the curved line is called a luminance gradation characteristic line. 
J5 Thereafter, in the luminance gradation correcting unit 1 9 T 1 6 gradation corrected luminance levels Lc(i) correspond- 

ing to pieces of gradation corrected histogram data of a luminance gradation corrected picture signal are produced 
from the pieces of summed normalized histogram data. 

20 Lc (') = Hnsn(i) 

In this case, because the inclination of the luminance gradation characteristic line is adjusted by the value K1 , the 
highest gradation corrected luminance level Lc(15) agrees with the upper limit luminance value Lmax of the dynamic 
range of the displaying unit 21 to set a ratio of the highest gradation corrected luminance level Lc(15) to the lowest 
25 gradation corrected luminance level Lc(0) within the dynamic range of the displaying unit 21 . 

Thereafter, a luminance gradation corrected picture signal, in which each piece of gradation corrected histogram 
data of the value H(i) corresponds to the gradation corrected luminance level Lc(i), is produced. Therefore, because 
a gradation corrected luminance level corresponding to a high luminance frequency represents a divided luminance 
range having a wide luminance width, a luminance contrast between a particular luminance level of a high luminance 
30 frequency and a group of neighboring luminance levels adjacent to the particular luminance level is emphasized in a 
luminance gradation corrected image indicated by the luminance gradation corrected picture signal. 

Thereafter, the luminance gradation corrected picture signal is processed in the main picture signal processing 
unit 20, and a gradation corrected image of the particular object is reproduced in the displaying unit 21 on condition 
that a ratio of the highest gradation corrected luminance level to the lowest gradation corrected luminance level is 
35 within a narrow dynamic range of the displaying unit .21 . 

Accordingly, a luminance contrast of a particular luminance level, in which a large number of pixels exist, for other 
luminance levels can be emphasized according to a conventional luminance gradation correcting method. Also, the 
highest gradation corrected luminance level Lc(15) agrees with an upper limit luminance value of the dynamic range 
of the displaying unit 21 according to a conventional luminance gradation correcting method to prevent that the lumi- 
40 nance values of the luminance gradation corrected picture signal exceeds the narrow dynamic range of the displaying 
unit 21. 

However, even though a low-luminance object such as a dark object is only photographed by the image pickup 
apparatus 1 1 and the highest luminance level of a synthesized picture signal indicating the dark object does not exceed 
an upper limit luminance value of the dynamic range of the displaying unit 21 , the luminance levels of the synthesized 
45 picture signal are unnecessarily adjusted to make the highest luminance level of the synthesized picture signal agree 
with the upper limit luminance value of the dynamic range of the displaying unit 21 . Therefore, there is a drawback that 
the user views a luminance gradation corrected image of the dark object at an unnatural feeling. 

An aim of the present invention is to provide, with due consideration to the drawbacks of such a conventional 
luminance gradation correcting method, a luminance gradation correcting method in which an image viewed by a user 
so at a natural feeling is reproduced even though the image is obtained by photographing a low-luminance object such 
as a dark object with an image pickup apparatus having a narrow dynamic range. 

According to the present invention there is provided: 

a luminance gradation correcting method in an image pickup apparatus having a dynamic range, comprising the 
ss steps of: 

detecting pieces of histogram data respectively indicating the number of pixels of a luminance level from an input 
picture signal produced in the image pickup apparatus, the input picture signal indicating a particular image; 
detecting a maximum luminance level from the luminance levels of the pieces of histogram data; 
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detecting a luminance gradation characteristic of the particular image indicated by the input picture signal from 
the pieces of histogram data corresponding to the luminance levels; 

producing a corrected luminance gradation characteristic of the particular image from the luminance gradation 
characteristic of the particular image according to the maximum luminance level and the dynamic range of the 

5 image pickup apparatus; 

correcting the luminance levels of the pieces of histogram data to a plurality of gradation corrected luminance 
levels according to the corrected luminance gradation characteristic of the particular image to set the gradation 
corrected luminance levels corresponding to the pieces of histogram data within the dynamic range of the image 
pickup apparatus; and s 

10 producing a luminance gradation corrected picture signal composed of the pieces of histogram data corresponding 

to the gradation corrected luminance levels to reproduce the particular image, in which each of the pixels has one 
corresponding gradation corrected luminance level, from the luminance gradation corrected picture signal. 

In the above steps, afterpieces of histogram data are detected from an input picture signal, a maximum luminance 

*5 level is detected from among the luminance levels, and a luminance gradation characteristic of the particular image is 
detected from the pieces of histogram data. A value of the piece of histogram data corresponding to the maximum 
luminance level is not equal to zero. Also, the luminance gradation characteristic denotes an inclination of a line indi- 
cated by a series of summed values of pieces of summed histogram data, and the summed values of the pieces of 
summed histogram data are determined by repeatedly calculating a sum of values of pieces of histogram data while 

20 the number of pieces of histogram data is increased one by one. 

Thereafter, a corrected luminance gradation characteristic of the particular image is produced from the luminance 
gradation characteristic of the particular image, and the luminance levels are corrected to a plurality of gradation cor- 
rected luminance levels according to the corrected luminance gradation characteristic. In this case, when the maximum 
luminance level is tower than or equal to a standard luminance level, the luminance levels are the same as the gradation 

25 corrected luminance levels. That is, any luminance gradation correction is not performed. When the maximum lumi- 
nance level is higher than the standard luminance level and is lower than or equal to an upper limit luminance value 
of the dynamic range, the corrected luminance gradation characteristic is set to make a maximum gradation corrected 
luminance level agree with the maximum luminance level. When the maximum luminance level is higher than the upper 
limit luminance value of the dynamic range, the corrected luminance gradation characteristic is set to make a maximum 

30 gradation corrected luminance level agree with the upper limit luminance value. 

Thereafter, a luminance gradation corrected picture signal composed of the pieces of histogram data corresponding 
to the gradation corrected luminance levels is produced , and the particular image, in which each of the pixels has one 
corresponding gradation corrected luminance level, is reproduced from the luminance gradation corrected picture sig- 
nal. 

35 Accordingly, even though the input picture signal is obtained by photographing a particular object such as a dark 

object of a low luminance, a light object of a high luminance or a combined object composed of a dark object portion 
and a light object portion, the particular image of the input picture signal can be viewed by a user at a natural feeling. 

The features and advantages of the present invention will be apparent from the following description of an exem- 
plary embodiment and the accompanying drawings, in which: 

40 

Fig. 1 is a block diagram of an image pickup apparatus operated according to a conventional luminance gradation 
correcting method; 

Fig. 2 shows a combining method for producing a synthesized picture signal from a long-time digital picture signal 
and a short-time digital picture signal; 
45 Fig. 3 shows a flow chart of a method for producing a luminance gradation characteristic of a synthesized image 

indicated by the synthesized picture signal according to the conventional luminance gradation correcting method; 
Fig. 4 is a block diagram of an image pickup apparatus operated according to a luminance gradation correcting 
method of the present invention; 

Fig. 5 shows a flow chart of a method for performing a luminance gradation correction to change luminance levels 
50 of pieces of histogram data of the synthesized picture signal to gradation corrected luminance levels; 

Fig. 6A shows a first, example of the synthesized picture signal in which the maximum luminance level is lower 
than or equal to a standard luminance level equivalent to half an upper limit luminance value of a dynamic range 
of a post-processing unit ol the image pickup apparatus shown in Fig. 4; 

Fig. 6B shows a second example of the synthesized picture signal in which the maximum luminance level is higher 
55 than the standard luminance level and is equal to or lower than the upper limit luminance value of the dynamic range; 

Fig. 6C shows a third example of the synthesized picture signal in which the maximum luminance level is higher 
than the upper limit luminance value of the dynamic range; 

Fig. 7A shows gradation corrected luminance levels of the first example shown in Fig. 6A; 
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Fig. 7B shows gradation corrected luminance levels of the second example shown in Fig. 6B; and 
Fig. 7C shows gradation corrected luminance levels of the third example shown in Fig. 6C. 

Preferred embodiments of a luminance gradation correcting method according to the present invention are de- 
scribed with reference to the drawings. 

Fig. 4 is a block diagram of an image pickup apparatus operated according to a luminance gradation correcting 
method of the present invention, and Fig. 5 shows a flow chart of a method for performing a luminance gradation 
correction to change luminance levels of. pieces of histogram data of the synthesized piclure signal to gradation cor- 
rected luminance levels. 

As shown in Fig. 4, an image pickup apparatus 31 comprises: the image pickup device 12 having M*N pixels, the 
pre-process ing unit 13, the analog-digital converter 14, the time-axis converter 15, the luminance level synthesizing 
unit 16, the histogram data detecting unit 17, 

a maximum luminance level detecting unit 32 for selecting a particular luminance levels L(i) corresponding pieces 
of particular histogram data having values H(i) not equal tozero from the pieces of original histogram data detected 
by the histogram data detecting unit 17, selecting a maximum luminance level L(Nmax) having the maximum 
luminance value among the particular luminance levels and detecting the number Nmax of the maximum luminance 
level; 

an inclination limitation selecting unit 33 for selecting a first inclination limitation in cases where the maximum 
20 luminance level is lower than or equal to a standard luminance level Ls, selecting a second inclination limitation 

in cases where the maximum luminance level is higher than the standard luminance level Ls and is equal to or 
lower than an upper limit luminance value Lmax of a dynamic range of a post-processing unit 38, and selecting a 
third inclination limitation in cases where the maximum luminance level is higher than the upper limit luminance 
value Lmax of the dynamic range, the standard luminance level Ls being half the upper limit luminance value Lmax 
25 of the dynamic range; 

a histogram data normalizing unit 34 for normalizing the values B(i) of the pieces of original h istogram data detected 
by the histogram data detecting unit 17 to produce values Hn1 (i) of pieces of normalized histogram data; 
an inclination limitation performing unit 35 for preparing a low limitation value Cpmin(i) and an upper limitation 
value cpmax(i) to satisfy Cpmin(i) = Cpmax(i) in cases where the first inclination limitation is selected, preparing 
a low limitation value Cpmin(i) and an upper limitation value Cpmax(i) to satisfy Cpmin(i) < Cpmax(i) in cases 
where the second inclination limitation or the third inclination limitation is selected, setting the value Hn1(i) of each 
piece of normalized histogram data to the low limitation value Cpmin(i) in case of Hn1(i) < Cpmin(i), setting the 
value Hnl (i) of each piece of normalized histogram data to the upper limitation value Cpmax(i) in case of Hnl(i) 
> Cpmax(i) and setting the value Hn1(i) of each piece of normalized histogram data as a value Hn2(i) (i = 0 to 
Nmax) of a piece of limited histogram data, an inclination of a curved line indicated by a series of summed values 
Hns1 (j) = EHn1 (i) (i = 0 to j) being limited in a curved line indicated by a series of summed values Hns2(j) = £Hn2(i); 
a luminance gradation characteristic producing unit 36 for accumulating the values Hn2(i) of the pieces of limited 
histogram data to produce summed values Hns2(j) = ZHn2(i) (i = 0 to j, j = 0 to Nmax) of pieces of limited histogram 
data and producing a limited inclination of a line indicated by a series of summed values Hns2(j) as a luminance 
40 gradation characteristic of the synthesized picture signal; 

a corrected luminance gradation characteristic producing unit 37 for multiplying each of the summed values Hns2 
(j) by a normalization factor K2=L(Nrnax)/Hns2(Nmax) to produce normalized summed values Hnsn2(j)=K2*Hns2 
(j) in cases where the first inclination limitation or the second inclination limitation is selected, multiplying each of 
the summed values Hns2(j) by a normalization factor K2=Lmax/Hns2(Nmax) to produce normalized summed val- 
ues Hnsn2(j)=K2*Hns2(j) in cases where the third inclination limitation is selected, and producing a corrected 
inclination of a line indicated by a series of normalized summed values Hnsn2(j) as a corrected luminance gradation 
characteristic of a luminance gradation corrected picture signal; 

a luminance gradation correcting unit 38 for setting the normalized summed values Hnsn2(j) as gradation corrected 
luminance values Lc(j) and producing a luminance gradation corrected picture signal classified into (Nmax + 1) 
pieces of gradation corrected histogram data of the values H(i) corresponding to the gradation corrected luminance 
levels Lc(i) (i = 0 to Nmax) to correct a luminance gradation of the synthesized image indicated by the synthesized 
picture signal to a luminance gradation indicated by the luminance gradation corrected picture signal, a luminance 
contrast between a particular gradation corrected luminance level Lc(n) and a group of neighboring luminance 
levels Lc(n-1) and Lc(n+1 ) adjacent to the particular luminance level being emphasized when an inclination of a 
ss line indicated by the series of summed value Hns2(n) at the luminance level L(n) is high; 

the main picture signal processing unit 20 for processing the luminance gradation corrected picture signal to pro- 
duce a luminance gradation corrected video signal; and 

a post-processing unit 39 for reproducing the particular object according to the luminance gradation corrected 
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video signal at a narrow dynamic range from the gradation corrected luminance level Lc(0) to the upper limit 
luminance value Lmax. 

In the above configuration, after a synthesized picture signal is produced in the luminance level synthesizing unit 

5 16, the values H(i) of the pieces of original histogram data are detected by the histogram data detecting unit 17 in the 
same manner as in the prior art shown in Fig. 3. 

Thereafter, as shown in Fig. 5, in the maximum luminance level detecting unit 32, a maximum luminance level L 
(Nmax) having the maximum luminance value is selected from particular luminance levels corresponding to pieces of 
original histogram data having values H(i) not equal to zero, and the number Nmax of the maximum luminance level 

to is detected in a step S201 . For example, as shown in Fig. 6A, the maximum number Nmax of the luminance level is 3 
in a first example in which a luminance value of the maximum luminance level L(Nmax) is equal to or lower than a 
standard luminance level Ls=L(4). The standard luminance level Ls is about half the upper limit luminance value Lmax 
of the dynamic range and is set to the low synthesized luminance lovel L3 shown in Fig. 2. Also, as shown in Fig. 6B, 
the maximum number Nmax of the luminance level is 4 in a second example in which a luminance value of the maximum 

15 luminance level L(N max) is higher than the standard luminance level Ls and is equal to or lower than an upper limit 
luminance value Lmax=L(5) of a dynamic range of the post -processing unit 39. Also, as shown in Fig 6C, the maximum 
number Nmax of the luminance level is 15 in a third example in which a luminance value of the maximum luminance 
level L(Nmax) is higher than the upper limit luminance value Lmax of the dynamic range. In this embodiment, three 
luminance gradation correction methods performed for the above examples differ from each other. 

20 Thereafter, an inclination limitation is selected in the inclination limitation selecting unit 33 in a step S202. In detail, 

a first inclination limitation is selected in cases where the maximum luminance level L(Nmax) is lower than or equal to 
the standard luminance level Ls, a second inclination limitation is selected in cases where the maximum luminance 
level L(Nmax) is higher than the standard luminance level Ls and is equal to or lower than the upper limit luminance 
value Lmax of the dynamic range, and a third inclination limitation is selected in cases where the maximum luminance 

25 level L(Nmax) is higher than the upper limit luminance value Lmax of the dynamic range. 

Also, in a step S203, thevalues H(i) of the pieces of original histogram data detected by the histogram data detecting 
unit 17 are normalized in the histogram data normalizing unit 34 to produce values Hn1(i) of pieces of normalized 
histogram data in the same manner as in the step S102 of Fig. 3. 

Thereafter, in a step S204, a value of original histogram data indicating an extremely large number of pixels is 

30 clipped to an upper-limitation value Cpmax(i), and a value of original histogram data indicating an extremely small 
number of pixels is clipped to a low limitation value Cpmin(i). In this embodiment, a luminance contrast between a 
particular luminance level, in which the number of pixels is large, and a group of neighboring luminance levels adjacent 
to the particular luminance level is emphasized by widening a divided luminance width of the particular luminance level. 
However, in cases where a divided luminance width of the particular luminance level is extremely widened by multiplying 

35 the divided luminance width by a gain determined according to an extremely large number of pixels corresponding to 
the particular luminance level, a slight difference between luminance levels of high luminance frequencies is excessively 
emphasized, so that a signal to noise (S/N) ratio deteriorates. Also, in cases where a divided luminance width of the 
particular luminance level is extremely narrowed by multiplying the divided luminance width by a gain determined 
according to an extremely small number of pixels corresponding to the particular luminance level, a luminance contrast 

40 between luminance levels of low luminance.frequencies is excessively suppressed. Therefore, an unnatural luminance 
gradation correction is undesirably performed. 

To prevent this unnatural luminance gradation correction in the inclination limitation performing unit 35 in the step 
S204, a low limitation value Cpmin(i) and an upper limitation value Cpmax(i) satisfying Cpmin(i) = Cpmax(i) are prepared 
in case of the first inclination limitation, a low limitation value Cpmin(i) and an upper limitation value Cpmax(i) satisfying 

45 cpmin(i) < Cpmax(i) is prepared in case of the second inclination limitation or the third inclination limitation, the value 
Hn1 (i) of each piece of normalized histogram data is set to the low limitation value Cpmin(i) in case of Hn1 (i) « Cpmin 
(i), the value Hn1 (i) of each piece of normalized histogram data is set to the upper limitation value Cpmax(i) in case of 
Hn1(i) » Cpmax(i), and the value Hn1 (i) of each piece of normalized histogram data is set as a value Hn2(i) (i = 0 to 
Nmax) of a piece of limited histogram data. Therefore, as shown in Figs. 6A to 6C, an inclination of a line indicated by 

50 a series of summed values Hns1 (j) = LHn1 (i) (i = 0 to j) is changed to a limited inclination of a line indicated by a series 
of summed values Hns2(j) = £Hn2(i). In this case, the upper limitation value Cpmax(i) indicates an upper limitation for 
the limited inclination, and the low limitation value cpmin(i) indicates a low limitation for the limited inclination. 

For example, the limitation values Cpmin(i) and Cpmax(i) are set to 32 for Nmax=511 in case of the first inclination 
limitation. In case of the second inclination limitation, the low limitation value Cpmin(i) is set to 32 (i=0,1) and 16 (i>2) 

55 for Nmax=511 , and the upper limitation value Cprnax(i) are set to 32 (i<4) and 16 (i>5) for Nmax=511. In case of the 
third inclination limitation, the low limitation value Cpmin (i) is set to 32 (i=0, £5) and 8 (1<i<4) for Nmax=511, and the 
upper limitation value Cpmax(i) are set to 32 (i<4) and 64(i>5)for Nmax=511. However, in Figs. 6A to6C, the limitation 
value Cpmin(i) is constant for all luminance levels L(i), and the limitation value Cpmax(i) is constant for alt luminance 
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levels L(i) for convenience. 

Thereafter, in a step S205, a luminance gradation characteristic of the pieces of limited histogram data is produced 
in the histogram data characteristic producing unit 36. That is, the values Hn2(i) of the pieces of limited histogram data 
are accumulated to produce summed values Hns2 (j). 

j 

Bns 2(j) = 2 Hn2(i) (j « 0 to Umax) 

i-0 

A limited inclination of a line indicated by a series of summed values Hns2(j) denotes the luminance gradation char- 
acteristic of the synthesized picture signal, as shown in Figs. 6A to 6C. 

Thereafter, in a step S206, a corrected luminance gradation characteristic of a luminance gradation corrected 
picture signal is produced in the luminance gradation characteristic producing unit 37, as shown in Figs. 6Ato6C. That 
is, each of the summed values Hns2(j) is multiplied by a normalization factor K2 to make the maximum gradation 
corrected luminance values Lc(Nmax) agree with the maximum luminance values L (Nmax) in cases where the first 
inclination limitation or the second inclination limitation is selected and to make the maximum gradation corrected 
luminance values Lc(Nmax) agree with the upper limit luminance value Lmax of the dynamic range in cases where the 
third inclination limitation is selected. 



Hnsn2(j) = K2 ' Hns2(j) (j = 0 to Nmax) 



K2 = L(Nmax)/Hns2(Nmax) (for the first or second inclination limitation) 



K2 = Lrnax/Hns2(Nmax) (for the third inclination limitation) 

In this case, an inclination of a line indicated by a series of summed normalized values Hnsn2(j) denotes the corrected 
luminance gradation characteristic of a luminance gradation corrected picture signal because Hnsn2(Nmax)=L(Nmax) 
is satisfied for the first inclination limitation or the second inclination limitation and Hnsn2(Nmax)=Lmax is satisfied for 
the third inclination limitation. Also, in cases where the first inclination limitation is selected, a difference between two 
summed normalized values Hnsn2(j) and Hnsn2(j+1) is constant because each value Hn2(i) is equal toCpmin(i) (or 
Cpmax(i)), and the value cpmin(i) (or Cpmax(i)) is set to make the difference agree with a luminance difference between 
two luminance levels L(j) and L(j+1). 

Thereafter, in a step S207, a luminance gradation of the synthesized image indicated by the synthesized picture 
signal is corrected in the luminance gradation correcting unit 38. That is, the normalized summed values Hnsn2(j) are 
set as gradation corrected luminance values Lc(j). 



Lc(j) = Hnsn2(j) (j = 0 to Nmax) 

Thereafter, in a step S208, a luminance gradation corrected picture signal composed of (Nmax + 1) pieces of 
gradation corrected histogram data corresponding to the gradation corrected luminance levels Lc(i) (i = 0 to Nmax) is 
produced. A value of each gradation corrected histogram data corresponding to one gradation corrected luminance 
level Lc(i) is equal to the value H(i) of one histogram data corresponding to one luminance level L(i). 

Therefore, when an inclination of the series of summed value Hns2(n) at a luminance level L(n) is high because 
a value Hn2(n) of the limited histogram data is high : a luminance contrast between a particular gradation corrected 
luminance level Lc(n) and a group of neighboring luminance levels Lc(n-1) and Lc(n+1) adjacent to the particular 
luminance level is emphasized. 

In this case, when the first inclination limitation is selected, because the difference between two summed normal- 
ized values Hnsn2(j) a nd Hnsn2(j+ 1 ) is constant and is set to a luminance width between two luminance levels adjacent 
to each other, any correction of the luminance gradation is not performed. That is, in cases where the maximum lumi- 
nance level L(Nmax) is lower than or equal to the standard luminance level Ls, the synthesized image indicated by the 
synthesized picture signal can be sufficiently reproduced in the post-processing unit 39 having a narrow dynamic range 
without any correction of the luminance gradation. 

Also, when the second inclination limitation is selected, because the synthesized image indicated by the synthe- 
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sized picture signal can be reproduced at a natural feeling in the post-processing unit 39 having a narrow dynamic 
range when a luminance contrast between a particular luminance level of a high luminance frequency and a group of 
neighboring luminance levels adjacent to the particular luminance level is emphasized, the luminance levels L(i) arc 
changed to gradation corrected luminance levels Lc(i) on condition that the maximum gradation corrected luminance 
5 level Lc(Nmax) agrees with the maximum luminance level L(Nmax). 

Also, when the third inclination limitation is selected, because a luminance range in the synthesized picture signal 
exceeds a dynamic range of the post-processing unit 39, it is required to narrow the luminance range of the synthesized 
piclure signal within Ihe dynamic range of the post-processing unit 39. Therefore, the luminance levels L(i) are changed 
to gradation corrected luminance levels Lc(i) on condition that the maximum gradation corrected luminance level Lc 
10 (Nmax) agrees with the upper limit luminance value Lmax of the dynamic range. 

Thereafter, the luminance gradation corrected picture signal is processed in the main picture signal processing 
unit 20, and an image of the particular object is reproduced in the post-processing unit 39 at a narrow dynamic range. 

Therefore, For example, in cases where the maximum luminance level L(Nmax) is lower than or equal to the 
standard luminance level Ls, the luminance levels L(i) of the original histogram data of the synthesized picture signal 
15 are changed to the gradation corrected luminance level Lc(i) shown in Fig. 7 A. Also, in cases where the maximum 
luminance level L(Nmax) is higher than the standard luminance level Ls and is equal to or lower than an upper limit 
luminance value of a dynamic range, the luminance levels L(i) of the original histogram data of the synthesized picture 
signal are changed to the gradation corrected luminance level Lc(i) shown in Fig. 7B. Also, in cases where the maximum 
luminance level L(Nmax) is higher than the upper limit luminance value Lmax of the dynamic range, the luminance 
20 levels L(i) of the original histogram data of the synthesized picture signal are changed to the gradation corrected 
luminance level Lc(i) shown in Fig. 7C. 

Accordingly, even though any object such as a dark object of a low luminance, a light object of a high luminance 
or a dark and high object composed of a dark object portion and a light object portion is photographed, an image viewed 
by a user at a natural feeling can be reproduced in an image pickup apparatus having a narrow dynamic range. 
25 Also, because the maximum gradation corrected luminance level in the luminance gradation corrected picture 

signal is determined according to a first relationship between the maximum luminance level of the synthesized picture 
signal and the standard luminance level Ls and a second relationship between the maximum luminance level and the 
upper limit luminance value Lmax of the dynamic range, the dynamic range of the post-processing unit can be effectively 
used even though any subject is photographed, an S/N ratio for an image of a low luminance portion can be improved 
30 while an image of a high luminance portion is reproduced. 

Having illustrated and described the principles of the present invention in a preferred embodiment thereof, it should 
be readily apparent to those skilled in the art that the invention can be modified in arrangement and detail without 
departing from such principles. We claim all modifications coming within the scope of the accompanying claims. 

35 

Claims 



1. A luminance gradation correcting method in an image pickup apparatus having a dynamic range, comprising the 
steps of: 

40 

detecting pieces of histogram data respectively indicating the number of pixels of a luminance level from an 
input picture signal produced in the image pickup apparatus, the input picture signal indicating a particular 
image; 

detecting a maximum luminance level from the luminance levels of the pieces of histogram data; 
45 detecting a luminance gradation characteristic of the particular image indicated by the input picture signal from 

the pieces of histogram data corresponding to the luminance levels; 

producing a corrected luminance gradation characteristic of the particular image from the luminance gradation 
characteristic of the particular image according to the maximum luminance level and the dynamic range of the 
image pickup apparatus; 

50 correcting the luminance levels of the pieces of histogram data to a plurality of gradation corrected luminance 

levels according to the corrected luminance gradation characteristic of the particular image to set the gradation 
corrected luminance levels corresponding to the pieces of histogram data within the dynamic range of the 
image pickup apparatus; and 

producing a luminance gradation corrected picture signal composed of the pieces of histogram data corre- 
55 sponding to the gradation corrected luminance levels to reproduce the particular image, in which each of the 

pixels has one corresponding gradation corrected luminance level, from the luminance gradation corrected 
picture signal. 
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A luminance gradation correcting method according to claim 1 in which the step of producing a corrected luminance 
gradation characteristic comprises the steps of. 

setting the corrected luminance gradation characteristic to the luminance gradation characteristic in cases 
where the maximum luminance level is lower than or equal to a standard luminance level which is half an 
upper limit luminance value of the dynamic range; 

setting the corrected luminance gradation characteristic to make a maximum gradation corrected luminance 
level among the gradation corrected luminance levels agree with the maximum luminance level in cases where 
the maximum luminance level is higher than the standard luminance level and is lower than or equal to the 
upper limit luminance value of the dynamic range; and 

setting the corrected luminance gradation characteristic to make a maximum gradation corrected luminance 
level among the gradation corrected luminance levels agree with the upper limit luminance value of the dynamic 
range in cases where the maximum luminance level is higher than the upper limit luminance value of the 
dynamic range. 

A luminance gradation correcting method according to claim 1 or 2 in which the step of detecting a maximum 
luminance level comprises the steps of: 

selecting one or more particular pieces of histogram data respectively having a value not equal to zero from 
the luminance levels: and 

detecting the maximum luminance level from one or more luminance levels corresponding to the particular 
pieces of histogram data. 

A luminance gradation correcting method according to claim 1, 2 or 3 in which the step of detecting a luminance 
gradation characteristic comprises the steps of: 

repeatedly calculating a sum of values of pieces of histogram data as a summed value of a piece of summed 
histogram data while the number of pieces of histogram data is increased one by one in the order of heightening 
a luminance level corresponding to one piece of particular histogram data added to the pieces of histogram 
data; and 

setting an inclination of a line indicated by the series of summed values of the pieces of summed histogram 
data, which are arranged in the order of increasing the number of pieces of histogram data corresponding to 
each piece of summed histogram data, as the luminance gradation characteristic. 

A luminance gradation correcting method according to claim 1 , 2, 3 or 4 in which the step of detecting pieces of 
histogram data comprises the steps of: 

detecting pieces of original histogram data respectively indicating the number of pixels of one luminance level 
from the input picture signal; 

preparing a limitation value in cases where the maximum luminance level is lower than or equal to the standard 
luminance level; 

preparing a low limitation value and an upper limitation value higher than the low limitation value in cases 
where the maximum luminance level is higher than the standard luminance level: 

setting the pieces of original histogram data to the pieces of histogram data having values equal to the limitation 
value in cases where the maximum luminance level is lower than or equal to the standard luminance level; 
setting each piece of original histogram data lower than the low limitation value to one piece of histogram data 
having a value equal to the low limitation value in cases where the maximum luminance level is higher than 
the standard luminance level; 

setting each piece of original histogram, data higher than the high limitation value to one piece of histogram 
data having a value equal to the high limitation value in cases where the maximum luminance level is higher 
than the standard luminance level; and 

setting each piece of original histogram data equal to or higher than the low limitation value and equal to or 
lower than the high limitation value to one piece of histogram data having the same value as that of the piece 
of original histogram data in cases where the maximum luminance level is higher than the standard luminance 
level. 

A luminance gradation correcting method according to claim 1, 2, 3, 4 or 5 in which the step of detecting pieces 
of histogram data includes the steps of: 
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producing a long-time picture signal indicating the particular image in a long exposure time; 
producing a short-time picture signal indicating the particular image in a short exposure time; and 
combining the long-time picture signal and the short-time picture signal to produce the input picture signal in 
which an image ot a low luminance object is indicated by the long-time picture signal and an image of a high 
luminance object is indicated by the short-time picture signal. 

An image pickup apparatus adapted to operate according to the method ot any one of the preceding claims. 
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FIG. 6B 
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FIG. 6C 
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FIG. 7 A 
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(57) A synthesized picture signal of a dynamic im- 
age is produced by combining a long-time picture signal 
representing a low luminance and a short-time picture 
signal representing a high luminance, pieces of histo- 
gram data corresponding to a plurality of luminance lev- 
els are detected from the synthesized picture signal, a 
sum of values of pieces of histogram data is repeatedly 
calculated as a summed value of a piece of summed 
histogram data while the number of pieces of histogram 
data is increased, and an inclination of a line indicated 
by a series of summed values is detected as a lumi- 
nance gradation characteristic. Thereafter, the lumi- 
nance gradation characteristic is corrected to a correct- 
ed luminance gradation characteristic according to a re- 
lationship between a maximum luminance level and an 
upper limit luminance level of a dynamic range, and a 
luminance gradation corrected signal composed of the 
pieces of histogram data corresponding to gradation 
corrected luminance levels is produced according to the 
corrected luminance gradation characteristic to make a 
maximum gradation corrected luminance level equal to 
or lower than the upper limit luminance level. Therefore, 
a low luminance object and/or a high luminance object 
can be clearly reproduced within the dynamic range. 
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THE NUMBER Nma* IS DETECTED. 
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VALUES OF ORIGINAL HISTOGRAM DATA ARE 
NORMALIZED 
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A LUMINANCE GRADATION CHARACTERISTIC 
IS PRODUCED 
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A LUMINANCE GRADATION OF A SYNTHESIZED 
PICTURE SIGNAL IS CORRECTED 



A LUMINANCE GRADATION CORRECTED PICTURE 
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